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Introduction 

English Language Learners (ELLs) are the fastest growing group of students in 
American public schools. According to Payan and Nettles (2008), the ELL population 
doubled in 23 states between 1995 and 2005. The U.S. Census Bureau estimates 
that by 2050, the Hispanic school-age population will exceed the non-Hispanic 
white school-age public school population (Fry & Gonzalez, 2008). Amidst these 
dramatic increases, ELL achievement remains among the lowest of all students. 
For example, on the 2009 National Assessment of Educational Progress (NAEP), 
72% of 8th-grade ELL students scored below basic in mathematics as compared to 
26% of non-ELL students (National Center for Education Statistics, 2009). Despite 
8 years of strong No Child Left Behind (NCLB, 2002) accountability provisions, ELL 
academic achievement remains one of the greatest challenges confronting states, 
school districts, and schools. 
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Drawing from a 3-year research effort funded by the U.S. Department of Education, 
UCLA’s National Center for Research on Evaluation, Standards, and Student 
Testing (CRESST) has developed a list of five priorities for improving the validity of 
assessment systems for ELL students. We define validity as the degree to which an 
assessment system produces accurate information about ELL students’ performance 
and provides a sound basis for policy decision-making. Our recommendations 
include improvements in: 

1. English Language Proficiency Standards and Assessments, 

2. ELL Classification and Reclassification, 

3. Content Assessments for ELL Students, 

4. ELL Test Accommodations, and 

5. Teacher Capacity and ELL Students' Opportunity to Learn. 

Recognizing that economic conditions across the United States are challenging and 
that states are often in different places on their ELL programs, we have prioritized 
each recommendation as 1, 2, or 3 (with 1 being the highest) and estimated resource 
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requirements as high, moderate, or low, to assist states 
and school districts in improving ELL assessment policies 
and practices. Recommendations are based on a series 
of CRESST research reports from the 3-year project and 
comprehensive input and feedback from state and school 
district ELL experts and policy makers at the 2009 Council 
of Chief State School Officers Assessment Conference. 
Research findings from the project that ied to the 
specific recommendations are described throughout this 
document (see Appendix A for the list of CRESST reports 
including the 3-year project research). We conclude with 
our recommendations for new research and an urgent call 
to action. 

We encourage states and school districts to use this 
document as a guide for discussion and action. Both an 
action guide (Appendix B) and a recommended readings 
list (Appendix C) are included at the end of this policy brief 
to help states and school districts discuss, evaluate, and 
improve the validity of their ELL accountability systems. 
As with all CRESST's work, we welcome your feedback and 
suggestions. 



1. English Language Proficiency 
Standards and Assessments 

A valid ELL assessment system begins with clear, high 
quality standards and close alignment between standards 
and assessments. Standards, including content and 
performance expectations, establish goals for curriculum, 
teaching, and learning, whereas assessments provide data 
on how well students are doing. For systems to work well, 
the two must be in sync. However, our research revealed 
that alignment, especially between state English language 
proficiency (ELP) standards and ELP assessments, 
frequently needed improvement. We found, for example, 
that some states have clear mismatches between the 
proficiency levels articulated in their ELP standards 
proficiency levels and those in their ELP assessments. 
Table 1 shows differences in both the number of levels 
and terminology across and within three states. Such 
misalignment may lead to confusion about learning goals 
and inaccurate interpretations by schools, districts, and 
states about ELL students’ degree of English acquisition. 



Table 1 

State ELP Standards and ELP Assessment Proficiency Levels for Three States 
(2006-2007 School Year) 



State 


State ELP standards 
proficiency ieveis 


State ELP assessment 
proficiency levels 


State 1 


(4 levels) 

Beginning, Intermediate, 
Advanced, Transitional 


(5 levels) 

Level 1, Level 2, Level 3, 
Level 4, Level 5 


State 2 


(4 levels) 

Beginning, 

Lower Intermediate, 
Upper Intermediate, 
Advanced 


(5 levels) 

Entering, Beginning, 
Developing, Expanding, 
Bridging 


State 3 


(6 levels) 

Pre-production, 

Early Production, 
Emergent, Intermediate, 
High Intermediate, 
Transitional 


(5 levels) 

Pre-emergent, Emergent, 
Basic, Intermediate, 
Proficient 
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Table 2 

Level 5 ELP Standards Level Definitions from Two States 



State 1 Advanced - Level 5 



State 2 Advanced - Level 5 



• Apply knowledge of sound/symbol relationships and 
basic word-formation rules to derive meaning from 
written text (e.g., basic syllabication rules, regular 
and irregular plurals, and basic phonics). 

• Apply knowledge of academic and social vocabulary 
while reading independently. 

• Be able to use a standard dictionary to find the 
meanings of unfamiliar words. 

• Interpret the meaning of unknown words by using 
knowledge gained from previously read text. 

• Understand idioms, analogies, and metaphors in 
conversation and written text. 



• Understand and obtain meaning from a wide range 
of texts available to native English speakers. 

• Read academic texts at the appropriate level. 
Understand a variety of literary genres. 

• Read and comprehend grammar and rhetorical 
features appropriate for the grade level. 

• Master strategies of reading comparable to native 
English-speaking students at their grade level. 

• Understand vocabulary that is basic and academic 
and be able to figure out technical vocabulary. 

• Read and interpret texts across the curriculum. 



We also found that definitions of proficiency and the levels 
through which proficiency developed varied widely across 
different states, making comparisons between states 
difficult. This finding also highlights the absence of any 
common academic English language framework on which 
states’ ELP standards should be based. Table 2 shows 
definitions from two states for their highest proficiency 
level on their respective ELP standards in the reading 
domain. The underlying standard from State 1 emphasizes 
decoding skills and strategies to find word meanings, 
while State 2 highlights the various genres and rhetorical 
features in academic texts. State 2 also emphasizes the 
understanding of different types of vocabulary. Which 
formulation best prepares ELL students for success is 
moot, as may be the relationship between ELP standards 
and the language competencies ELL students need for 
success in academic content. 

Consequently, we suggest the following: 

Recommendation 1.1: ELP standards and ELP 
assessments should be aligned with each other 
and with the related content standards. 

(Priority: 2, Resources: Moderate) 

Detail: Valid ELL assessment systems begin with rigorous 
state standards that drive all parts of learning, including 
state content standards, ELP standards, ELP assessments, 
instruction, classroom assessment, and professional 



development. Close alignment of these components 
helps schools to know if ELL students are meeting or not 
meeting the standards. As illustrated in Table 1, however, 
improvement is needed to align the descriptions and the 
number of proficiency levels between ELP standards and 
an ELP assessment within a state. As the No Child Left 
Behind Act of 2001 (NCLB, 2002) specifies, the alignment 
between ELP and content standards should be taken into 
consideration in this alignment improvement work. By 
doing so, schools and teachers can more efficiently use 
the assessment results to adjust instruction or provide 
interventions for ELL students’ English development. 

Recommendation 1.2: ELP standards should include 
essential academic English language requirements. 
(Priority: 2, Resources: Moderate) 

Detail: There is widespread consensus that academic 
language proficiency is one of the key factors in 
determining students’ success in schools. Although states 
have developed their ELP standards with an emphasis 
on academic English language development, academic 
English is variously defined as seen in Table 2. Our 
research also revealed substantial variation in the degree 
and complexity of academic English presented in two 
ELP assessments. As a substantial body of theory and 
research is now available on the nature and development 
of academic English, states need to revisit their ELP 
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standards to ensure their consistency with available 
knowledge (Bailey, 2007; Chamot & O’Malley, 1994; 
Scarcella, 2003; Schleppegrell, 2001). 

2. ELL Cl assification and Reclassification 

Because assessment data are used to draw inferences 
about ELL performance and often to compare performance 
across time and locales, the adequacy and consistency 
of ELL classification and reclassification are key issues 
to consider in improving the validity of ELL assessment 
systems. Figure 1 shows a typical process. 




Figure 1. Common procedure for identifying and 
reclassifying ELL students (source: Wolf et al., 2008). 

Our review of policies and practices revealed that states 
often used very different processes to identify and reclassify 
ELL students. For example, although there was a trend for 
states to use a single evaluation tool (i.e., ELP assessment) 
to identify English language learners, our research found 
that 16 states allowed local school districts to base the 
decision on a locally selected language assessment from 
an approved state list. Such practices, while supporting 
local flexibility, raise major comparability issues both 
within and across states. 



Migrant students, who are oftentimes ELL students as well, 
may be especially impacted by lack of standardization. 
Migrant ELL students may be identified or reclassified in 
one school district or state, for example, but not identified 
or reclassified in another, merely due to inconsistent state 
criteria. Reporting may be negatively affected and migrant 
ELL students may receive inappropriate or inadequate 
services. 

Similarly, for reclassifying ELL students, we found that 
states often used a number of different criteria, ranging 
from ELP test scores, content test scores, school 
personnel input, parent or guardian input, student grades, 
interviews, and observations. Figure 2 shows, for example, 
that 14 states used just one criterion, an ELP assessment, 
to reclassify students as English proficient. Eleven states 
used two criteria, usually test scores from both the ELP 
assessment and the content assessment. Eighteen states 
allowed individual districts to set their own reclassification 
criteria, which included subjective decisions. Such diverse 
methods inevitably lead to inconsistent identification and 
reclassification of many students, thus posing a central 
threatto the validity of the accountability system. Solutions 
are straightforward and should have low resource impact, 
leading to clearer understanding for all constituencies in 
the identification and reclassification processes. 




Figure 2. Number of criteria states use for ELL 
reclassification decisions, 2006-2007 school year. 



Recommendation 2.1: States should clearly define 
ELL terms and be consistent in their use. 

(Priority: 2, Resources: Low-Moderate) 

Detail: Different terms and definitions for ELL students 
(e.g., non-English proficient, limited English proficient, 
language minority, or reclassified fluent English learner) 
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may imply different understandings of and attitudes 
toward ELL students. Simple, consistent definitions and 
descriptions should be used in all documents, which can 
help practitioners implement ELL policies appropriately. 
In particular, the definition for reclassified students 
should be clear so that practitioners understand what it 
means to be a reclassified student. Clear definitions will 
help the right ELL students receive the instruction and 
accommodations they need, while increasing the validity 
of reports. 

Recommendation 2.2: States should evaluate and 
standardize their identification, classification, and 
reclassification criteria for ELL students. 

(Priority: 1, Resources: Moderate) 

Detail: State policy and supporting documents should 
specify the methods and criteria for identifying, classifying, 
and reclassifying ELLs. If multiple criteria are used, policy 
guidelines should specify how each criterion should be 
employed. Policies should specify whether and how the 
criteria may be modified by local districts due to their 
specific needs. Additionally, states should require local 
districts to document the criteria used to make decisions 
for each student. 

Recommendation 2.3: States should examine their 
reclassification rates and evaluate the effects of 
their reclassification policies. 

(Priority: 2, Resources: Moderate) 

Detail: We found that different reclassification policies 
were associated with different ranges in achievement 
gaps between ELL and non-ELL students. For example, 
reclassified students in states that required ELL students 
to reach the highest proficiency levels on their ELP test 
in order to exit ELL status often performed higher than 
their non-ELL peers on content assessments. On the other 
hand, reclassified students in states with more lenient 
ELL exit standards performed comparably to other ELL 
students on content assessments. We note that a strict 
reclassification policy may generate more long-term ELL 
students who have been identified as ELL for over 3 years. 
Few state ELP reclassification programs focused on the 
underlying causes and possible solutions for students 
who fail to become reclassified as fully English proficient. 
Without clear reclassification policies and adequate school 
or classroom support, ELL students may be retained in 
English learning remedial programs and have limited 



access and time for other challenging courses. This 
may delay graduation and have negative postsecondary 
consequences. 

States and districts need to closely analyze ELL student 
performance on both content and ELP assessments for 
exiting or close-to-exiting students as well as long-term 
ELL students. We recommend that states, districts, and 
schools regularly use data systems to track reclassification 
rates and evaluate whether their reclassification policies 
have positive or negative consequences for reclassified 
and long-term ELL students. States should revise policies 
if necessary. 

Recommendation 2.4: States should create a 
longitudinal database system to provide validity 
evidence for ELL assessments and to support 
appropriate decisions for ELL students. 

(Priority: 2, Resources: Moderate) 

Detail: Longitudinal databases to support ELL 

identification and reclassification decisions should include 
unique student identification numbers that are used 
consistently across schools. Databases should contain 
detailed background information (e.g., native language, 
level of English language proficiency, instructional history, 
mobility, socioeconomic status) to enable investigations of 
the validity and effects of decisions from the assessments 
across ELL subgroups. The data should link individual 
student and teacher IDs so that student performance 
can be linked to teacher professional development and 
can guide the improvement of instruction for students’ 
specific needs. 

3. Content Assessments for ELL Students 

Content assessments — for example, state assessments of 
math, science, or English language arts — are developed 
for the purpose of measuring knowledge, skills, and 
understanding of academic content (i.e., the intended 
construct). A fundamental validity concern for any 
assessment is that it measures the intended construct(s) 
and is not confounded by constructs that are not the 
intended target of the assessment. Because it requires 
at least some language skills, even a math content test 
administered in English is therefore also a test of English 
skills for ELL students. Consequently, the language 
demands of any test may get in the way of ELL students 
showing what they know and inappropriately constrain 
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their performance. The following recommendations will 
lead to more valid content assessment results for ELL 
students. 

Recommendation 3.1: States should include explicit 
item-writing rules and specific principles in their 
content test Requests for Proposals (RFPs), 
specifying that test developers avoid unnecessary 
linguistic complexity on test items. 

(Priority: 2, Resources: Moderate) 

Detail: In one of our studies, we examined language 
characteristics in three states’ mathematics and science 
assessments. Despite the limited amount of text in the 
test items, we found a broad and varied range of linguistic 
complexity among the three states. Some assessments 
were linguistically complex, while others were less so. The 
item-writing guidelines and principles adopted by each 
state were associated with the linguistic complexity of 
that state’s items. 

Figure 3 illustrates how a math test item may be 
linguistically modified to increase ELL understanding 
(Abedi, 2009). The original version begins, “A certain 
reference file contains approximately...” The modified 
version simplifies the language to, “Mack’s company 
sold...” The modified version is linguistically easier 
to understand without changing the construct being 
assessed. Research indicates that linguistic modification 
helps performance of other low-performing students, not 
just ELL students (Abedi, 2006). 



Original: 

A certain reference file contains 
approximately six billon facts. 
About how much millions is that? 


Modified: 

Mack's company sold six billion 
pencils. About how many millions is 
that? 


A. 6,000,000 


A. 6,000,000 


B. 600,000 


B. 600,000 


C. 60,000 


C. 60,000 


D. 6,000 


D. 6,000 


E. 600 


E. 600 



Figure 3. Example of linguistically modified test item 
(Abedi, 2009) 

Recommendation 3.2: Test validation shouid inciude 
procedures to identify and reduce potential item bias in 
content items. (Priority: 2, Resources: Low-Moderate) 

Detail: Whereas Recommendation 3.1 applies to the test 
developmental stage, this recommendation calls for review 



and empirical study of ELL performance during field and/ 
or operational tests. We recommend that content reviews 
of linguistic complexity be combined with differential item 
functioning analyses for ELL students to identify items 
that may be biased against ELLs, that is, those items 
that ELL students miss at a substantially higher rate than 
non-ELL students of the same ability level. Biased items 
should be carefully examined and revised or replaced as 
appropriate. Our research results clearly indicated that 
easy items (based on non-ELL students’ performance) with 
more complex language are likely to function differentially 
against ELL students (Wolf & Leon, 2009). States and/ 
or test developers should make efforts to examine ELL 
students’ performance and the language characteristics 
in their content assessments. 

4. ELL Test Accommodations 

Test accommodations are changes intended to make 
content tests more accessible for students who otherwise 
would face obstacles in showing what they know but 
without changing the construct(s) measured by the test. 
For ELL students, test accommodations — changes in 
test administration settings, administration procedures, 
and/or tasks — are aimed at reducing the confounding 
of language and content knowledge and skills on content 
tests administered in English. Accommodations are 
intended to “level the playing field” for ELL students, 
helping them take the same tests as mainstream 
students. However, our studies raised concerns on the 
use of some accommodations, including the consistency 
of accommodation use and the comparability of 
accommodated and non-accommodated scores. We make 
the following suggestions to increase the appropriate and 
valid use of accommodations. 

Recommendation 4.1: States should provide 
comprehensive accommodation guidelines based 
on current research for selecting and using 
ELL accommodations. Assign responsibility of 
disseminating guidelines to a specific person. 

(Priority: 2, Resources: Low-moderate) 

Detail: Although many states have made progress in 
developing and producing accommodation guidelines 
for ELL students, our case studies and examination 
of existing state and local school documentations and 
databases revealed great variability in guidelines and 
practices. We found that detailed state guidelines were 
associated with better school and teacher knowledge of 




CRESST Policy Brief 1 0 Full Report :: Spring 2010 :: 7 



accommodation policies and procedures, as well more 
consistent implementation and compiiance with state 
provisions. Detailed guidelines can help increase the 
comparability of ELL results to non-ELL students. 

But just having guidelines is not enough. In our studies, 
we found that most schools did not have an ELL specialist 
or coordinator. These schools often had difficulty following 
sound accommodation practices, resulting in incorrect or 
even no accommodations provided to ELL students. State 
policy accommodations should specify: 

Who is eligible for accommodations? We found that 
many teachers did not know who was eligible for 
accommodations. When teachers clearly know 
who is eligible for accommodations, they usually 
provide similar accommodations during instruction 
and classroom assessment, thereby helping ELL 
students prepare for state-level tests. 

Who makes accommodation decisions? We recommend 
that a team of both content and ELL teachers 
be the decision makers who meet regularly to 
identify students’ needs and assign appropriate 
accommodations. 

Accommodations assignment criteria. State guidelines 
should specify useful assignment criteria to local 
decision makers, based on students’ native language, 
levels of English language proficiency, content test 
results, and instructional practice. For example, 
one of our studies found a positive accommodation 
effect when it was administered to ELL students who 
had acquired content knowledge and had previous 
experience with the given accommodation. Such 
research-based examples could help inform specific 
state guidelines. 

Allowabie and prohibited accommodations. States 
should clearly specify both allowable and prohibited 
accommodations by content area and grade level. 
Clear definitions of each accommodation and 
its implementation procedures should also be 
included. Our case studies revealed, for example, 
that read-aloud accommodations were often 
interpreted differently by teachers. Some teachers 
read aloud just the test directions, others read 
aloud part of each test question, and still others 
read aloud the complete test. Providing a clear, 
operational definition and a detailed description 
about administration procedures will help teachers 
use the accommodations in a standardized way, 
resulting in enhanced comparability. 



A key person, such as the testing coordinator or the ELL 
speciaiist, to help teachers and school administrators 
accurately implement accommodations. This 
individual should be well informed of state or 
district accommodation policies and monitor local 
implementation. 

Recommendation 4.2: States and districts shouid 
conduct reguiar professionai deveiopment meetings 
to inform both content and ELL teachers about 
appropriate accommodations use. 

(Priority: 2, Resources: Moderate) 

Detail: Comprehensive accommodation guidelines 

will be of limited value if teachers are not provided 
with appropriate training to implement the guidelines. 
Our research findings showed that few teachers, 
especially content teachers, were provided sufficient 
opportunities to understand and implement existing 
state ELL accommodation policies. This insufficiency 
was particularly true for content teachers. Professional 
development should include information on the latest 
ELL assessment and accommodation research findings in 
order to promote the valid use of accommodations linked 
to empirical evidence of what and how accommodations 
best work for ELL students. 

Recommendation 4.3: State guidelines should require 
systematic recording on a statewide database of 
specific accommodations provided to individual 
students. Such data can be used to evaluate the 
validity and efficacy of accommodations. 

(Priority: 2, Resources: Moderate-High) 

Detail: Our studies found that states lacked adequate 
records of the accommodations provided to individual 
students on states’ standards-based content assessments. 
We also found a substantial mismatch between state and 
local school records of accommodations administered to 
individual ELL students. As part of the test administration 
procedures, schools also should document who was 
involved in accommodation decisions and the accommo- 
dations each ELL student received. 

Entry of accommodations data in a statewide database 
(see Recommendation 2.4) can promote transparency 
across different levels of reporting (i.e., state, district, 
and school) and can function to increase the accuracy of 
school accommodation records. Accessible to teachers, 
the database should contain ELL characteristics (e.g., 
native language, English language proficiency levels. 
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mobility) and the types of accommodations provided to 
each student. Estabiishing systematic recording practices 
will provide valuable information to improve the validity 
of assessing ELL students’ content knowledge and skills. 
The information will also help states identify successful 
and less successful accommodation strategies for specific 
student groups and individual students. 

Recommendation 4.4: States and districts shouid 
monitor the use of accommodation guideiines 
reguiariy to ensure consistent appiication of 
accommodation poiicies. 

(Priority: 3, Resources: Moderate-High) 

Detail: An effective accommodations monitoring process 
can help policy makers evaluate and improve local 
schools’ accommodation usage and further inform 
policies and guidelines. Regular monitoring will boost 
active implementation of accommodation guidelines, 
improve data-recording practices, and promote 
comparable data from one locale to the next. As part of 
the monitoring process, states not only need to analyze the 
accommodation data, examining the extent to which the 
individualized accommodations are implemented during 
testing, but also have occasional meetings with district 
personnel to gather information on the effectiveness of 
the implemented policies. 

5. Teacher Capacity and ELL Students' 
Opportunity to Learn 

Accountability systems are intended to support the 
improvement of learning. The quality of ELL students’ 
opportunity to learn the knowledge and develop the skills 
that will be assessed and teachers’ capacity to meet 
student needs, while not primary emphases in our studies, 
are fundamental to such improvement. In addressing the 
validity of assessment for ELL students, some of our 
study findings also have implications for (a) the nature 
of the curriculum and instruction that can engage ELL 
students and (b) for teacher capacities that need to be 
developed. 

Recommendation 5.1: Encourage schools to use an 
integrated approach to developing content and English 
language proficiencies, simultaneously teaching 
academic language and content knowledge. 

(Priority: 1, Resources: Moderate) 

Detail: Our study findings demonstrated that many state 



content assessments contain academic vocabulary that 
may be difficult for ELL students. If students do not 
understand the questions in an assessment because they 
don’t understand academic language, inferences about 
students’ knowledge and skills may be invalid. Schools 
should integrate academic language instruction along 
with content instruction to improve assessment validity 
and increase ELL students’ opportunity to learn. 

Recommendation 5.2: Encourage and build teachers’ 
capacity to use appropriate ELL accommodations 
during classroom instruction and assessment. 

(Priority: 2, Resources: Moderate) 

Detail: When appropriately and effectively used, testing 
accommodations are an important way to increase 
the validity of content assessment for ELL students. 
Accommodations also offer an essential strategy for 
providing ELL students access to content curriculum that 
is delivered in English (i.e., without accommodations, ELL 
students’ language skills may limit their ability to benefit 
from content curriculum and instruction). Consistent with 
previous research, our findings imply that even direct 
linguistic-support accommodations (e.g., English glossary 
and reading aloud the entire test) do not help ELL students 
on state content tests if students have not previously used 
the same accommodations in classrooms. It seems self 
evident as well that without appropriate linguistic support 
in curriculum and instruction, ELL student learning is 
likely to be highly compromised. 

Study findings that content teachers often have little 
knowledge about ELL accommodations give significant 
pause. Inadequate knowledge translates into infrequent 
use of accommodations during classroom content 
instruction or assessment. Implications for teacher 
capacity building seem clear. 

Recommendation 5.3: Pre-service teacher 
education and post-service professionai deveiopment 
shouid expand and integrate ELL assessment and 
accommodation knowledge and strategies. 

(Priority: 3, Resources: High) 

Detail: Teacher survey and interview responses from our 
study sample suggested that both content teachers and 
ELL specialists often lacked sufficient knowledge about 
successful ELL assessment and accommodation strategies. 
We also found a lack of sufficient communication between 
these two important groups. With the rapid growth of ELL 
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populations, states should place a substantial focus on 
increasing teacher knowledge of current ELL issues and 
appropriate assessment and accommodation strategies 
including pre-service teacher education and continuing 
teacher education. Program goals should include adequate 
time for cross-articulation of successful strategies 
between content teachers and ELL specialists. 

Future Research Agenda 

Continuing research on assessment and instruction of 
ELL students must be conducted to provide empirical 
evidence that supports and improves current reform 
efforts. We particularly call for more rigorous research on 
the following eight topics to improve both the validity and 
quality of ELL assessment systems. Longitudinal studies 
that track the same ELL students across multiple years are 
likely to provide more information for long-term solutions 
than short-term, cross-sectional studies. 

Topic 1. ELL Policy. 

Although NCLB has been in effect since 2002 and places 
strong emphasis on adequate yearly progress for all 
students including ELL students, very little research 
has directly examined NCLB’s effect on ELLs. Looking 
at whether and how current programs are affecting ELL 
students’ learning is essential to improving current policy 
and practices. But solid studies will require resolution 
of some of the issues addressed in this policy brief. 
Additional research in this area would likely lead to 
improved educational policies, both nationally and at state 
levels, which in turn could lead to improved outcomes for 
ELL students. 

Topic 2. Expand the current empirical research on 
constructs of academic English language. 

A comprehensive, operational definition of academic 
English has yet to be developed. Questions include what 
unique vocabulary, grammatical features, and language 
functions constitute academic English? Is there a common 
developmental sequence? New research in this area should 
examine the academic language demands of various 
subject areas and grade levels. Research should examine 
stages of academic English language development and 
implications for curriculum, instruction, and assessment. 



Topic 3. Continue validity research on ELP assessment 
uses, especially score reporting. 

Our research revealed that states using the same ELP 
assessment often employed different scoringand reporting 
systems. For instance, some states used a composite 
score of four modality scores (listening, speaking, reading, 
and writing) equally whereas some states weighted each 
modality score differently to create a composite score. 
Different weighting and reporting methods will produce 
different decisions related to ELL students. Additional 
research would increase our knowledge regarding the 
extent of such practices and potentially lead to more 
consistent and valid practices. 

Topic 4. Develop research-based item-writing 
guidelines for accessible and fair test construction for 
ELL students. 

We discussed earlier that item-writing rules could reduce 
unnecessary linguistic complexity in content assessments. 
Research that examines which writing rules and how the 
writing rules are most effective would provide helpful 
information to states and assessment developers alike. 
The research could also lead to principles for guiding 
the language characteristics of content assessments. 
For instance, Abedi, Courtney, and Leon (2003) discuss 
several principles including the use of high frequency 
words, avoiding colloquial and double-meaning words, 
and reduction of unnecessary expository materials for 
mathematics assessments. 

The concept of Universal Design has been promoted for 
the assessment of students with disabilities but may also 
hold promise for ELL students (Thompson, Johnstone, 
& Thurlow, 2002). New types of assessments, including 
computer and game-based strategies, may also work for 
ELL students. Designing assessments that are accessible 
to the greatest number of students possible could reduce 
the need for accommodations. 

Topic 5. Expand accommodations research. 

Despite numerous studies on the effects of 
accommodations, some findings are inconclusive and 
provide limited evidence to assure valid procedures for 
selecting and applying appropriate accommodations. Our 
review studies found that no state ELL accommodation 
manuals in 2006-2007 described research-based 
rationale or use for accommodations. Consequently, 
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rigorous research in both assessment construction 
(Topic 4) and accommodations should be a high priority. 
Findings will likely lead to improved ELL accountability 
policies, increased system validity, and improvements in 
instructional practice. 

Topic 6. Use the variation across states to examine the 
vaiidity and consequences of different ELL policies and 
practices. 

Our research found substantial variance in ELL policies 
and performance between states and school districts. For 
example, the magnitudes of 8th-grade math achievement 
gaps between current ELLsand non-ELLs were considerable 
and the size of the gaps varied across the three sample 
states. Patterns of performance reclassified ELLs — that 
is, those designated as English proficient — also varied. 
Causes for the differences in gaps are elusive. Additional 
research will help identify the important variables that 
influence disparate performance. We also found high 
variance in the methods that states and school districts 
use to identify and reclassify ELL students. Examining 
the validity and consequences of ELL identification and 
reclassification criteria would be part of this research. The 
end goal is to make better decisions about ELL students 
and the schools that serve them. 

Topic 7. Examine the characteristics, conditions, and 
programmatic variables that are associated with ELL 
success and failure, especially for long-term ELL 
students. 

A significant proportion of ELL students never exits ELL 
status and continues to lag behind their peers. For example, 
in California more than 50-60% of ELL students remain 
in ELL status for more than 10 years (Grissom, 2004; 
Parrish et al., 2006) with continued low-performance 
compared to their reclassified peers (Kim & Flerman, 
2009). Consequently, there is a dire need for research to 
identify methods to help long-term ELL students acquire 
English language proficiency, exit ELL status, and increase 
academic achievement. 

There is also limited empirical research on the assessment 
and instructional needs for ELL students at different 
levels of English proficiency. Like many student groups, 
ELL students are heterogeneous, with a broad range of 
performance. Consequently, differential support may need 
to be provided to ELL students based on ELL strengths 
and needs. This research topic would investigate specific 



classroom support and assessment methods for different 
levels of ELL students. 

Topic 8. Investigate ELL students’ opportunity to learn 
including classroom materials. 

The low performance of ELL students on content 
assessments is sometimes attributed to a lack of 
opportunity to learn. We have anecdotal evidence, for 
example, that some ELL students are being taught with 
below-grade level materials. Such practices may be based 
on a belief that ELL students will find such materials 
easier to understand due to ELL language and/or content 
knowledge deficiencies. But this practice may prevent 
ELL students from reaching the same high standards 
expected of all students. Opportunity to learn research 
could help better define what constitutes adequate 
opportunity to learn, examine the types of opportunities 
to learn (including classroom materials) affecting ELL 
achievement, and determine what specific tools effectively 
measure opportunity to learn. The research would likely 
expand the availability and quality of tools for monitoring 
students’ opportunities to learn and improve teacher 
instruction. 

Call to Action 

The ELL performance gap is one of the most challenging 
issues confronting American education. NAEP scale scores 
of ELL students in reading, for example, were virtually 
unchanged between 2002 and 2007. In both years, 8th- 
grade ELL students scored 41 points lower than their 
non-ELL counterparts. The persistent achievement gap 
has negative implications for ELL students. For example, 
in New York City only 23.6% of ELL students graduate 
from high school within 4 years (Zehr, 2009), which has 
lifelong consequences. To address these critical problems, 
we encourage policy makers to work closely with their 
staffs and other ELL experts to implement many of the 
recommendations in this policy brief. The enclosed Action 
Guide for Improved ELL Accountability Systems (Appendix 
B) is intended to assist states in this process, with a 
realistic focus on resources and capacity. 

While as a nation we have greatly expanded our knowledge 
about ELL instruction and assessment, a great deal of 
work remains. As Adam Urbanski, Vice-President of the 
American Federation of Teachers stated, “If we always do 
what we’ve always done, we will get what we’ve always 
got.” 
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APPENDIX B 



ACTION GUIDE FOR IMPROVED ELL ACCOUNTABILITY SYSTEMS 

The following action guide is intended to help states and school districts discuss, evaluate, and improve the validity of their 
ELL accountability systems. Page 14 includes space for listing either state or school district improvement goals. 



Recommendations 


Status 


Action 

Needed 


Estimated 
Completion Date 


Comments 


Recommendation 1.1. ELP Standards 
and Assessments: Alignment 

ELP standards and ELP assessments 
are aligned with each other and with 
the related content assessments. 


□ Level 1 ■ Planning 

□ Level 2 ■ Early work in progress 

□ Level 3 ■ Einal work in progress 

□ Level 4 ■ Complete 








Recommendation 1.2. ELP Standards 
and Academic English 

ELP standards include essential 
academic English language 
requirements. 


□ Level 1 ■ Planning 

□ Level 2 ■ Early work in progress 

□ Level 3 ■ Final work in progress 

□ Level 4 ■ Complete 








Recommendation 2.1. ELL 
Classification and Reclassification: 
Definitions 

State provides clear definitions of 
ELL students, including reclassified/ 
exited students. 


□ Level 1 ■ Planning 

□ Level 2 ■ Early work in progress 

□ Level 3 ■ Final work in progress 

□ Level 4 ■ Complete 








Recommendation 2.2. ELL 
Classification and Reclassification: 
Criteria 

State has evaluated and standardized 
its identification, classification, and 
reclassification criteria of ELL students. 


□ Level 1 ■ Planning 

□ Level 2 ■ Early work in progress 

□ Level 3 ■ Final work in progress 

□ Level 4 ■ Complete 








Recommendation 2.3. ELL 
Classification and Reclassification: 
Data Review 

State uses data annually to evaluate 
reclassification rates and policies. 


□ Level 1 ■ Planning 

□ Level 2 ■ Early work in progress 

□ Level 3 ■ Final work in progress 

□ Level 4 ■ Complete 








Recommendation 2.4. ELL 
Classification and Reclassification: 
Longitudinal Database 

State has a longitudinal database 
system that provides validity evidence 
supporting classification and 
reclassification decisions. 


□ Level 1 ■ Planning 

□ Level 2 ■ Early work in progress 

□ Level 3 ■ Final work in progress 

□ Level 4 ■ Complete 








Recommendation 3.1. Content 
Assessment: Item-Writing Rules 

State has explicit item-writing rules 
and principles for avoiding unnecessary 
linguistic complexity on test items. 


□ Level 1 ■ Planning 

□ Level 2 ■ Early work in progress 

□ Level 3 ■ Final work in progress 

□ Level 4 ■ Complete 








Recommendation 3.2. Content 
Assessment: Validation for ELLs 

Test validation procedures examine 
the language demands of test items on 
content assessments. 


□ Level 1 ■ Planning 

□ Level 2 ■ Early work in progress 

□ Level 3 ■ Final work in progress 

□ Level 4 ■ Complete 
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APPENDIX B (continued) 



Recommendations 


Status 


Action 

Needed 


Estimated 
Completion Date 


Comments 


Recommendation 4.1. Test 
Accommodations: Guidelines 

State has comprehensive accommodation 
guidelines for selecting and using ELL 
accommodations. 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 








Recommendation 4.2. Test 
Accommodations: Use 

State and/or districts regularly include 
appropriate accommodation practices in 
their professional development programs. 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 








Recommendation 4.3. Test 
Accommodations: Recordkeeping 

State records accommodations use in a 
statewide database. 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 








Recommendation 4.4. Test 
Accommodations: Evaluation 

State regularly evaluates district use of 
accommodation guidelines. 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 








Recommendation 5.1. Teacher Capacity 
and Opportunity to Learn: Integrated 
Approach 

State policy encourages schools to use an 
integrated curriculum and instructional 
approach. 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 








Recommendation 5.2. Teacher Capacity 
and Opportunity to Learn: Classroom 

State policy encourages teachers to use 
appropriate ELL accommodations during 
classroom instruction and assessment. 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 








Recommendation 5.3. Teacher Capacity 
and Opportunity to Learn: Knowledge and 
Strategies 

Pre-service teacher education and post- 
service professional development includes 
ELL assessment and accommodation 
knowledge and strategies. 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 








*State/District Improvement Goal #1 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 








*State/District Improvement Goal #2 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 








*State/District Improvement Goal #3 


□ Level 1 - Planning 

□ Level 2 - Early work in progress 

□ Level 3 - Final work in progress 

□ Level 4 - Complete 









We have included space for states and school districts to Include their own ELL accountability system goals. 
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